Syntactic Normalization Of Spontaneous Speech

This paper presents some techniques that provide a standard parsing system for the analysis of ill-formed utterances. These techniques are feature generalization and heuristically driven deletions. PROBLEM Generally the development of grammars, formalisms and natural language processors is based on written language data or, sometimes, not real data at all, but invented 'example sentences'. This holds for both computational and general linguistics. Thus many parsing systems that work quite well for sentences like la. and lb. fail, if they get applied to the authentic data in 2a. and 2b.: la. lb. 2a. 2b. die Grundform ist nicht eckig the basic form is not angular das blaue habe ich als Waage auf dem gr0nen llegen I have got the blue one lying upon theOAT greenOAT OneDAT like a balance die die Grund die Grundform sind is nich Is nleh eckig the the basic the basic form are is not is not angular das blaue hab ich ale Waage aul das gr0ne liegen I have got the blue one lying upon theACC greenAcc oneACC like a balance To native recipients the utterances in 2. appear to be more or less defective, but interpretable expressions. Moreover, the interpretation of 2a. or 2b. might require even less effort than, for instance, understanding an absolutely grammatical 'garden path sentence'. Since utterances like 2a. and 2b. occur quite frequently in spontaneous speech, an approach to parsing everyday language has to provide techniques that cover repairs, ungrammatical repetitions (2a.), case-assignment violation (2b.), agreement errors and other phenomena that have been summarized under the label 'iU-formed' in earlier research (Kwasny/Sondheimer "1 am indebted to Dafydd Gibbon, Hans Karlgren and Hannes Rieser for their comments on earlier drafts of this paper. Though the present paper will adhere to this terminology , it should be emphasized that it is not presupposed that there are any general criteria precise enough to tell us exactly whether some utterance is 'ill-formed' relative to a natural language. Let us assume, instead, that some utterance U is 'ill-formed (defective, irregular ....) with respect to a grammar G' iff U is not a sentence of the language specified by G. Since, for instance, repairs exhibit a high degree of structural regularity (el. Schegloff et al. 1977, Lever 1983, Kindt/Laubenstein in preparation) one might prefer to describe them within the grarxmaar and not within some other domain (e.g. within a pro-duction/perception model). Therefore the concept 'ill-formed' is …